Image processing device and image processing method

ABSTRACT

An image processing device including a subject frame setting section which, by operating a subject detector which detects a subject captured in an image, sets a subject frame which surrounds a predetermined range of the subject detected from the image; an acceptance frame setting section which sets an acceptance frame with a range wider than the subject frame according to the context of the image; a position detecting section which detects a specified position on an image which is specified by a user; and a recognizing section which recognizes a subject which is a tracking target based on the acceptance frame set by the acceptance frame setting section and the specified position detected by the position detecting section.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation of U.S. Ser. No. 14/043,964, filedOct. 2, 2013 which is a continuation of U.S. Ser. No. 13/109,281 filedMay 17, 2011 (now U.S. Pat. No. 8,643,740) which claims the benefit ofpriority under 35 U.S.C. §119 from Japanese Patent Application No.2010-151951, filed Jul. 2, 2010. The entire contents of which areincorporated herein by reference.

BACKGROUND

The present disclosure relates to an image processing device and animage processing method, and in particular to an image processing deviceand an image processing method where it is possible to recognize asubject which is a tracking target with precision.

In the past, in digital still cameras, digital video cameras, and thelike (referred to appropriately below as digital cameras), a functionfor tracking a subject such as the face of a certain person selected bya user or an object other than a face was mounted. Then, it is possibleto continuously maintain various parameters (for example, focus,brightness, and the like) of the digital camera to be optimal accordingto the state of the position of a tracking subject.

As a method for the user to select a subject to be photographed using adigital camera, there is a method for selecting by touching a subjectdisplayed on a touch panel, a method for selecting using a guide framefor matching a subject with the angle of view in advance, and the like.

For example, in Japanese Unexamined Patent Application Publication No.2006-101186, an imaging device is disclosed in which, when the userwants to track a subject other than a face, a dedicated mode in which itis possible to detect the subject is set to operate, and a subjecttracking function is enabled.

SUMMARY

However, in the digital camera in the related art, if the user is notable to accurately designate the subject which is the tracking target,it is difficult for the subject which is intended by the user to berecognized with precision.

It is desirable to be able to recognize a subject which is a trackingtarget with precision.

An image processing device according to an embodiment of the disclosureis provided with a subject frame setting section which, by operating asubject detector which detects a subject captured in an image, sets asubject frame which surrounds a predetermined range of the subjectdetected from the image; an acceptance frame setting section which setsan acceptance frame with a range wider than the subject frame accordingto the context of the image; a position detecting section which detectsa specified position on an image which is specified by a user; and arecognizing section which recognizes a subject which is a trackingtarget based on the acceptance frame set by the acceptance frame settingsection and the specified position detected by the position detectingsection.

An image processing method according to the embodiment of the disclosureincludes setting, by operating a subject detector which detects asubject captured in an image, a subject frame which surrounds apredetermined range of the subject detected from the image; setting anacceptance frame with a range wider than the subject frame according tothe context of the image; detecting a specified position on an imagewhich is specified by a user; and recognizing a subject which is atracking target based on the acceptance frame and the specifiedposition.

According to the embodiment of the disclosure, the subject frame whichsurrounds the predetermined range of the subject detected from the imageis set by operating the subject detector which detects the subjectcaptured in the image and the acceptance frame with a range wider thanthe subject frame is set according to the context of the image. Then,the specified position on the image which is specified by the user isdetected and the subject which is the tracking target is recognizedbased on the acceptance frame and the specified position.

An image processing device according to another embodiment of thedisclosure is provided with a subject frame setting section which, byoperating a subject detector which detects a subject captured in animage, sets a subject frame which surrounds a predetermined range of thesubject detected from the image; an acceptance frame setting sectionwhich sets an acceptance frame with a range wider than the subject frameaccording to an imaging mode set for performing imaging using imagingconditions which are appropriate for a certain image; a positiondetecting section which detects a specified position on an image whichis specified by a user; and a recognizing section which recognizes asubject which is a tracking target based on the acceptance frame set bythe acceptance frame setting section and the specified position detectedby the position detecting section.

An image processing method according to the other embodiment of thedisclosure includes setting, by operating a subject detector whichdetects a subject captured in an image, a subject frame which surroundsa predetermined range of the subject detected from the image; setting anacceptance frame with a range wider than the subject frame according toan imaging mode set for performing imaging using imaging conditionswhich are appropriate for a certain image; detecting a specifiedposition on an image which is specified by a user; and recognizing asubject which is a tracking target based on the acceptance frame and thespecified position.

According to the other embodiment of the disclosure, the subject framewhich surrounds the predetermined range of the subject detected from theimage is set by operating the subject detector which detects the subjectcaptured in the image and the acceptance frame with a range wider thanthe subject frame is set according to the imaging mode set forperforming imaging using the imaging conditions which are appropriatefor a certain subject. Then, the specified position on the image whichis specified by the user is detected and the subject which is thetracking target is recognized based on the acceptance frame and thespecified position.

According to the embodiments of the disclosure, it is possible torecognize the subject which is the tracking target with precision.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram illustrating a configuration example of adigital camera according to an embodiment of the disclosure;

FIG. 2 is a diagram illustrating a screen example when an imaging modeis a person mode;

FIG. 3 is a diagram illustrating a screen example when an imaging modeis a pet mode;

FIG. 4 is a diagram illustrating another screen example when an imagingmode is a pet mode;

FIG. 5 is a diagram illustrating a setting example of a touch acceptanceframe according to an imaging mode;

FIG. 6 is a diagram illustrating a setting example of a touch acceptanceframe according to the size of a subject;

FIG. 7 is a diagram illustrating a setting example of a touch acceptanceframe according to the movement of a subject;

FIG. 8 is a diagram illustrating a setting example of a touch acceptanceframe according to the movement of a subject;

FIG. 9 is a block diagram illustrating a configuration example of atracking subject recognition section;

FIG. 10 is a flow chart describing a process of starting subjecttracking;

FIG. 11 is a flow chart describing a touch acceptance frame settingprocess;

FIG. 12 is a diagram describing an example where a tracking subject isrecognized according to an imaging mode; and

FIG. 13 is a block diagram illustrating a configuration example of acomputer according to an embodiment of the disclosure.

DETAILED DESCRIPTION OF EMBODIMENTS

Below, specific embodiments of the disclosure will be described indetail while referring to the diagrams.

FIG. 1 is a block diagram illustrating a configuration example of adigital camera according to an embodiment of the disclosure.

In FIG. 1, a digital camera 11 is configured to be provided with a lens12, an imaging section 13, a display section 14, an external I/F(interface) 15, and an image processing section 16.

The lens 12 is configured by, for example, a group of a plurality oflenses which provided a zoom function and the like, and light from asubject is concentrated and an image of a subject is produced on animaging surface of an imaging element of the imaging section 13. Theimaging section 13 is configured by, for example, a CCD (Charge CoupledDevice), a CMOS (Complementary Metal Oxide Semiconductor) sensor, or thelike, and images the image of the subject and outputs the image obtainedvia the imaging to the image processing section 16.

The display section 14 is configured by an LCD (Liquid Crystal Display),an organic EL (Electro Luminescence) display, or the like, and the imagewhich is imaged in the imaging section 13 and output by imagingprocessing being carried out in the image processing section 16, thatis, the image where the subject is surrounded by a subject frame, isdisplayed. The external I/F 15 is configured by a terminal or the likefor outputting the image output from the image processing section 16 to,for example, an external display (not shown).

The image processing section 16 carries out image processing with regardto the image imaged by the imaging section 13 and outputs the image tothe display section 14 and the external I/F 15. The image processingsection 16 performs image processing where, for example, a subject (forexample, a face) which is captured in an image is detected and a subjectframe which surrounds a predetermined range of the subject issynthesized in the image. In addition, when an arbitrary subject isselected by a user, the image processing section 16 performs a subjecttracking process where the subject is tracked and performs a processwhere various types of parameters are adjusted so each is optimal whenthe subject which is a tracking target (referred to below as trackingsubject) is imaged.

The image processing section 16 is configured to be provided with animage memory 21, a mode setting section 22, a touch detection section23, a subject frame setting processing section 24, a tracking subjectrecognition section 25, a detector switching section 26, and a subjecttracking processing section 27.

In the image memory 21, data of the image imaged by the imaging section13 is sequentially supplied, and the image memory 21 sequentially storesthe supplied data of the image.

The mode setting section 22 sets the various types of parameters used inthe imaging mode specified by a user. The digital camera 11 is providedwith, for example, various imaging modes such as a person mode whereimaging is performed using imaging conditions which are appropriate forimages of people, a pet mode where imaging is performed using imagingconditions which are appropriate for images of animals, and the like. Inaddition, the imaging mode which is set in the mode setting section 22is referenced in processes which are performed by the subject framesetting processing section 24 and the tracking subject recognitionsection 25.

The touch detection section 23 detects a position (specified position)which is touched when a user touches the screen of the display section14 based on a signal from a touch panel (not shown) which is installedso as to overlap with the screen of the display section 14. Then, thetouch detection section 23 supplies information showing the touchposition to the tracking subject recognition section 25.

The subject frame setting processing section 24 has a plurality ofdetectors which detect a subject captured in an image and performs asubject frame setting process where the size of a subject frame, whichsurrounds a predetermined range of the subject detected by thedetectors, is set. For example, as shown in FIG. 1, the subject framesetting processing section 24 has a face detector 28-1 and an animaldetector 28-2 and sets the subject frame with regard to the face of aperson detected by the face detector 28-1, the face of an animaldetected by the animal detector 28-2, or the like. In addition, thesubject frame set by the subject frame setting processing section 24 issynthesized in the image and is displayed in the display section 14.

The tracking subject recognition section 25 recognizes the trackingsubject based on the subject frame set by the subject frame settingprocessing section 24 when the information showing the touch positiontouched by a user is supplied from the touch detection section 23. Forexample, the tracking subject recognition section 25 recognizes thesubject which is set in the subject frame as the tracking subject if thetouch position touched by the user is at an inner side of the subjectframe. Then, the tracking subject recognition section 25 supplies theinformation where the tracking subject is specified to the detectorswitching section 26 in a case where it is possible to recognize thetracking subject.

The detector switching section 26 recognizes the detector used by thesubject frame setting processing section 24 in the detection of thetracking subject via the tracking subject recognition section 25 whenthe information specifying the tracking subject is supplied from thetracking subject recognition section 25. Then, the detector switchingsection 26 performs a process of supplying the information specifyingthe tracking subject to the subject tracking processing section 27 andswitching the detector used by the subject tracking processing section27 in the subject tracking process to the detector used in the detectionof the tracking subject.

The subject tracking processing section 27 starts the subject trackingprocess where the tracking subject which is specified by the informationwhich specifies the tracking subject is tracked using the detector whichis switched by the detector switching section 26. According to this, itis possible for the subject tracking processing section 27 to executethe subject tracking process using the detector which is appropriate forthe tracking subject.

In addition, in the information specifying the tracking subject, forexample, information which shows the position of the tracking subject,information which shows color and luminance of the tracking subject, andthe like are included. When the data of the next image is stored in theimage memory 21, the subject tracking processing section 27 detects aregion, which has color and luminance close to that of the trackingsubject in a predetermined region in the vicinity of the position of thetracking subject specified in the previous image, in the next image andspecifies the tracking target. Then, the subject tracking processingsection 27 repeatedly performs specification of the tracking subjectfrom the image sequentially stored in the image memory 21 based on theposition of the tracking subject specified by the image and the colorand the luminance of the tracking subject. In this manner, the subjecttracking process is performed and the subject tracking processingsection 27 adjusts the various types of parameters so that each isoptimal when the tracking subject is imaged.

In this manner, in the digital camera 11, the subject which has becomethe tracking target is recognized by the user touching the screen of thedisplay section 14 and the subject tracking process where the subject istracked is performed.

Here, for example, in a case where a plurality of the subject frames isset with regard to an image where a plurality of subjects is captured,it is not possible for the tracking subject recognizing section 25 torecognize the tracking subject in a case where the touch positiontouched by the user is not at an inner side of any of the subjectframes. In this case, it is possible for the tracking subjectrecognizing section 25 to reference the imaging mode set by the modesetting section 22 and recognize the tracking subject.

A process where the tracking subject recognizing section 25 referencesthe imaging mode and recognizes the tracking subject will be describedwith reference to FIGS. 2 and 3.

Screens P11 to P13 of the display section 14 when the person mode is setas the imaging mode are shown in FIG. 2, and screens P14 to P16 of thedisplay section 14 when the pet mode is set as the imaging mode areshown in FIG. 3. In addition, in the digital camera 11, the facedetector 28-1 normally operates in the subject frame setting processingsection 24.

As shown in FIG. 2, in the screens P11 to P13 of the display section 14when the imaging mode is the person mode, an icon showing a person and“person mode” are displayed. Then, when two children and a dog areimaged as the subject shown in screen P11, the subject frame settingprocessing section 24 sets the subject frame to the faces of the twochildren detected by the face detector 28-1. According to this, as shownscreen P12, a subject frame F11 and a subject frame F12 set in regard tothe faces of the two children are displayed.

Here, the position where the user touches the screen of the displaysection 14 is an outer-side vicinity of the subject frame F11 such asthe tip of the finger mark shown in screen P12 and is in the vicinity ofthe dog. In this case, since the imaging mode is the person mode, thetracking subject recognizing section 25 infers that the subject whichthe user wants as the tracking target is the person even in an outerside of the subject frame F11, and the face of the child where thesubject frame F11 is set is recognized as the tracking subject.According to this, as shown in screen P13, a subject frame F11′ for thetracking subject is displayed. The color, size, and the like from theprevious subject frame F11 which was recognized as the tracking targetis changed and the subject frame F11′ for the tracking subject isdisplayed.

On the other hand, as shown in FIG. 3, in screens P14 to P16 of thedisplay section 14 when the imaging mode is the pet mode, an iconshowing an animal and “pet mode” are displayed. In screen P14, an imagewith the same configuration as screen P11 of FIG. 2 is displayed, and asshown in screen P15, a location which is in an outer-side vicinity ofthe subject frame F11 and in the vicinity of the dog (the same locationas in screen P12 of FIG. 2) is touched by the user.

In this case, since the imaging mode is the pet mode, the trackingsubject recognizing section 25 infers that the subject which the userwants as the tracking target is the animal and not the children even inan outer side of the subject frame F11 and the dog is recognized as thetracking subject. According to this, as shown in screen P16, a subjectframe F13′ for the tracking subject which surrounds a predeterminedregion of the face of the dog is displayed.

Here, when the imaging mode is the pet mode, both the face detector 28-1and the animal detector 28-2 operate at the same time. In this manner,in a case where the plurality of detectors in the digital camera 11operates at the same time, it is possible to display the subject frameswith regard to the subjects which are detected by the respectivedetectors.

That is, as shown in FIG. 4, in screen P15′ of the display section 14when the pet mode is set as the imaging mode, it is possible to displaythe subject frame also on the face of the dog in the same manner as thefaces of the children.

However, in order for the tracking subject recognizing section 25 todetermine that the position touched by the user on the screen of thedisplay section 14 is an outer-side vicinity of the subject frame, it isnecessary that the region which stipulates the vicinity is determined.As the region, other than setting a wide frame with a constant spacingin a fixed manner with regard to the subject frame, for example, a framewhere the spacing changes may be set according to the imaging target,the imaging mode, or the like. In this manner, the frame, where it isacceptable for the subject which is set in the subject frame, to betouched by the user, even when the position where the user touches thescreen of the display section 14 is an outer-side of the subject frame,is appropriately referred to below as the touch acceptance frame.

For example, in FIG. 5, screen P17 of the display section 14 when theperson mode is set as the imaging mode and screen P18 of the displaysection 14 when the pet mode is set as the imaging mode are shown.

Screens P17 and P18 are where a person and a dog are imaged and have thesame configuration. In addition, in screens P17 and P18, a subject frameF14 with regard to the face of the person and a subject frame F15 withregard to the face of the dog are set as the same locations. Inaddition, in screens P17 and P18, a touch acceptance frame F16 withregard to the subject frame F14 and a touch acceptance frame F17 withregard to the subject frame F15 are shown with dashed lines. Here, thetouch acceptance frames F16 and F17 are diagrammatically shown in orderfor description and are not shown in the display section 14.

As shown in screen P17, in the case where the imaging mode is the personmode, since the subject which is prioritized in imaging is the person,the touch acceptance frame F16 where there is the setting of the subjectframe F14 of the face of the person which is the subject appropriate tothe imaging mode is set to be larger than the touch acceptance frame F17where there is the setting of subject frame F15 of the face of the dog.

On the other hand, as shown in screen P18, in the case where the imagingmode is the pet mode, since the subject which is prioritized in imagingis the animal, the touch acceptance frame F17 where there is the settingof the subject frame F15 of the face of the dog which is the subjectappropriate to the imaging mode is set to be larger than the touchacceptance frame F16 where there is the setting of subject frame F14 ofthe face of the person.

In this manner, by setting the touch acceptance frame with a sizeaccording to the imaging mode, it is possible for the digital camera 11to estimate the selection intention of the user even if the touch of theuser is separated from the subject and to recognize the subject inaccordance with the intention as the tracking subject.

In addition, as shown in screen P17, in regard to the person which isthe subject, the spacing of the touch acceptance frame F16 with regardto the subject frame F14 is set so that the spacing on the lower sidewith regard to the subject frame F14 is wider than the spacing on theupper side, right side, and left side with regard to the subject frameF14. This is because, for example, in the case where the subject is theperson, the capturing of the body of the person in the lower side of thesubject frame (face frame) often occurs. In this manner, by setting thespacing on the lower side of the touch acceptance frame F16 with regardto the subject frame F14 to be wide, it is also possible to set theperson as the tracking subject when the body of the person captured inthe image is touched.

In addition, as shown in screen P17, in the case where the subject isthe animal, the touch acceptance frame F17 is set so as to have equalspacing with regard to the subject frame F15. In this manner, in thedigital camera 11, the size of the touch acceptance frame according tothe type of the subject, specifically, the size of the touch acceptanceframe according to the type of detector which detects the subject is setin advance.

Furthermore, the size of the subject frame is adjusted according to thesize of the subject captured in the image and the size of the touchacceptance frame is adjusted according to the size of the subject frame.

That is, as shown in FIG. 6, in a case where a subject A is captured asso to be larger than a subject B, a subject frame F18 which is set forthe subject A is set to be larger than a subject frame F20 which is setfor the subject B. In addition, a touch acceptance frame F19 which isset for the subject A is also set to be larger than a touch acceptanceframe F21 which is set for the subject B.

In addition, the touch acceptance frame F19 is set so that the spacing(margin) on the lower side of the touch acceptance frame F19 with regardto the subject frame F18 is wider than the spacing on the upper side,right side, and left side, so as to include the body of the subject B.In the same manner, the touch acceptance frame F21 is set so that thespacing on the lower side with regard to the subject frame F20 is wideso as to include the body of the subject B.

Furthermore, the spacing of the touch acceptance frame with regard tothe subject frame may be adjusted, for example, according to themovement of the subject, according to the context (scene) of thesubject.

That is, as shown in FIG. 7, when the subject A is captured alone, whenmovement is detected where the subject B is approaching the subject Afrom the left side of the subject A, the spacing on the left side of thetouch acceptance frame F19 (the subject B side) which is set for thesubject A is adjusted so as to become narrower. After that, whenmovement where the subject B separates from the subject A is detected,the touch acceptance frame F19 is adjusted to return to the originalspacing. Here, although not expressed in FIG. 7, the spacing on thesubject A side of the touch acceptance frame F21 which is also set forthe subject B is adjusted so as to become narrower as the subject Bapproaches the subject A and is adjusted to return to the originalspacing when the subject B separates from the subject A.

In addition, for example, as shown in FIG. 8, when the subject A iscaptured alone, movement where the subject B is approaching the subjectA from the left side of the subject A and movement where a subject C isapproaching the subject A from the right side of the subject A aredetected. In this case, the spacing on both sides of the touchacceptance frame F19 which is set for the subject A is adjusted so as tobecome narrower. After that, when movement where the subject B and thesubject C separate from the subject A is detected, the touch acceptanceframe F19 is adjusted to return to the original spacing. Here, the touchacceptance frame F21 which is set for the subject B and a touchacceptance frame F23 which is set for the subject C are adjustedaccording to the movement of the subject B and the subject C.

As is described with reference to FIGS. 5 to 8, the touch acceptanceframe is set so as to surround a wider range than the subject frame, andthe size of the touch acceptance frame is adjusted due to the imagingmode and the imaging scene (the size, type, movement, and the like ofthe subject).

Next, FIG. 9 is a block diagram illustrating a configuration example ofthe tracking subject recognition section 25 which sets the touchacceptance frame.

In FIG. 9, the tracking subject recognition section 25 is configured tobe provided with a storage section 31, a movement detection section 32,a touch acceptance frame setting processing section 33, a touchdetermination section 34, and a recognition section 35.

In the storage section 31, information relating to the subject frame setby the subject frame setting processing section 24 is supplied, andinformation relating to the subject frame is stored in the storagesection 31 for a predetermined period. In the information relating tothe subject frame, information which shows the position and the size ofthe subject frame and information which shows the type of the detectorwhich detects the subject are included.

The movement detection section 32 detects movement of the subject bydetermining a change in position of the subject frame in a predeterminedperiod based on the information relating to the subject frame stored inthe storage section 31.

The touch acceptance frame setting processing section 33 sets the touchacceptance frame with regard to the subject frame based on the imagingmode set by the mode setting section 22 and the information relating tothe subject frame stored in the storage section 31. In addition, thetouch acceptance frame setting processing section 33 adjusts the size ofthe set touch acceptance frame based on movement of the subject framewhich is detected by the movement detection section 32.

When the information which shows the touch position touched by the useris supplied from the touch detection section 23, the touch determinationsection 34 determines whether or not the touch position is in the rangeof the touch acceptance frame which is set by the touch acceptance framesetting processing section 33. Then, in a case where it is determinedthat the touch position is in the range of the touch acceptance frame,the touch determination section 34 decides that the subject where thetouch acceptance frame is set is the tracking subject and notifies therecognition section 35.

Here, in a case where it is determined that the touch position is not inthe range of the touch acceptance frame, the touch determination section34 operates the other detectors with regard to the subject frame settingprocessing section 24. For example, when only the face detector 28-1 isoperated, there is a case where the user touches the subject other thanthe person, for example, the animal. At this time, it is determined thatthe touch position is not in the range of the touch acceptance frame,but by operating the animal detector 28-2 using the subject framesetting processing section 24, it is possible to detect the animal whichis touched by the user and it is possible to recognize the animal as thetracking subject.

The recognition section 35 recognizes the tracking subject when there isnotification of the tracking subject from the touch determinationsection 34 and outputs information which specifies the tracking subject.

Next, FIG. 10 is a flow chart describing a process where the digitalcamera 11 starts subject tracking.

For example, the process starts when the digital camera 11 is activatedand the imaging using the imaging section 13 starts, and in step S11,the subject frame setting processing section 24 starts operating theface detector 28-1. Then, the subject frame setting processing section24 starts the subject frame setting process where the subject frame(face frame) is set with regard to the face of the person which isdetected from the image stored in the image memory 21, and the processprogresses to step S12.

In step S12, the touch detection section 23 determines whether or notthe user touches the screen of the display section 14 based on thesignal from the touch panel of the display section 14. Then, the processwaits until the touch detection section 23 determines that the user hastouched the screen of the display section 14, and when it is determinedthat the user has touched the screen of the display section 14, theprocess progresses to step S13.

In step S13, the touch detection section 23 outputs the information(coordinates) which shows the touch position touched by the user to thetracking subject recognition section 25 and the tracking subjectrecognition section 25 acquires the touch position.

After the process of step S13, the process progresses to step S14, thetouch acceptance frame setting process (refer to FIG. 11 describedlater) is performed, and the touch acceptance frame is set with regardto the subject frame. After the process of the touch acceptance framesetting process, the process progresses to step S15.

In step S15, in the tracking subject recognition section 25, the touchdetermination section 34 determines whether or not the touch position isin the range of the touch acceptance frame based on the touch positionacquired in step S13.

In a case where it is determined in step S15 that the touch position isnot in the range (outside of the range) of the touch acceptance frame,the process progresses to step S16, and the touch determination section34 determines whether or not all of the detectors of the subject framesetting processing section 24 are operating.

In a case where it is determined in step S16 that not all of thedetectors of the subject frame setting processing section 24 areoperating, the process progresses to step S17.

In step S17, the touch determination section 34 requests operation ofthe detectors other than the detectors which are operating with regardto the subject frame setting processing sector 24. In accordance withthis, the subject frame setting processing section 24 starts theoperation of the other detectors, the process returns to step S14, andbelow, the process is repeated in the same manner. That is, thedetection of the subject using the detectors which have begun operatingis performed, the subject frame is set when the subject is newlydetected, and the touch acceptance frame is newly set with regard to thesubject frame.

On the other hand, in a case where it is determined in step S16 that allof the detectors of the subject frame setting processing section 24 areoperating, the process returns to step S12, and below, the process isrepeated in the same manner. That is, in this case, it is not possiblefor the tracking subject to be recognized with regard to the touch ofthe user, and the detection of a new touch is awaited. Here, in thiscase, that it was not possible to recognize the tracking subject may bedisplayed on the display section 14.

On the other hand, in a case where it is determined in step S15 that thetouch position is in the range of the touch acceptance frame, theprocess progresses to step S18.

In step S18, the recognition section 35 recognizes the subject, which isset in the touch acceptance frame in which the touch position is in therange, as the tracking subject and acquires the color and luminance inthe subject frame set for the subject. Then, the recognition section 35supplies information, which includes information which shows theposition of the tracking subject, and information which shows the colorand the luminance of the tracking subject and the like and whichspecifies the tracking subject, to the detector switching section 26.

After the process of step S18, the process progresses to step S19, thedetector switching section 26 performs a process of recognizing thedetector which detects the tracking subject and switching the detectorwith regard to the subject tracking processing section 27 so as to usethe detector in the subject tracking process. In addition, the detectorswitching section 26 supplies the information which specifies thetracking subject to the subject tracking processing section 27 and theprocess progresses to step S20.

In step S20, the subject tracking processing section 27 uses thedetector switched in step S18 and the subject tracking process where thetracking subject is tracked starts based on the information whichspecifies the tracking subject. After the process of step S20, theprocess returns to step S12, the process waits until the user touchesthe next subject, and below, the process is repeated in the same manner.

As above, in the digital camera 11, when the region in an inner side ofthe touch acceptance frame is touched, since the subject which is set inthe touch acceptance frame is recognized as the tracking subject, it ispossible for the subject to be recognized with precision even when thetouch position of the user is outside of the subject frame.

In addition, in the case where the position touched by the user isoutside of the range of the touch acceptance frame, by operating theother detectors, it is possible to detect the subject even when thedetector for detecting the subject which the user wants as the trackingtarget is not operating. According to this, when setting the subjectwhich is the tracking target, it is possible to improve the convenienceof use for the user.

That is, in the case where the imaging mode is the person mode and onlythe face detector 28-1 is operating, when the user touches the animalwhich is displayed in the display section 14, the operation of theanimal detector 28-2 automatically starts and the animal is recognizedas the tracking subject. In this manner, it is no bother for the user tochange the imaging mode to the animal mode and it is possible to startthe subject tracking process with the animal as the tracking subject.

In addition, when the plurality of detectors is operated, the trackingsubject is recognized, and the subject tracking process is started inthis manner, it is possible for the detector switching section 26 tostop the operation of the detectors other than the detector used in thedetection of the tracking subject. According to this, it is possible toreduce the processing which is executed in the image processing section16, and for example, it is possible to suppress the use of power.

Next, FIG. 11 is a flow chart describing the touch acceptance framesetting process of step S14 of FIG. 10.

In step S31, the touch acceptance frame setting processing section 33reads out and acquires the information, which relates to the subjectframe displayed in the display section 14 at a timing when the touchdetermination section 34 acquires the touch position, from the storagesection 31.

After the process of step S31, the process progresses to step S32, thetouch acceptance frame setting processing section 33 references the modesetting section 22, the imaging mode which is currently set is acquired,and the process progresses to step S33.

In step S33, the touch acceptance frame setting processing section 33recognizes the type of the subject where the subject frame is set basedon the information relating to the subject frame acquired in step S31.Then, the touch acceptance frame setting processing section 33 sets thetouch acceptance frame with regard to the subject frame in a size whichis a standard registered in advance based on the type of the recognizedsubject and the imaging mode acquired in step S32.

Here, in the touch acceptance frame setting processing section 33, thesize, which is the standard of the touch acceptance frame with regard tothe size of the subject frame, is registered in advance. In addition,the size of the touch acceptance frame with regard to the subject framediffers in accordance with the type of the subject for each of theimaging modes as described with reference to FIG. 5, and in the touchacceptance frame setting processing section 33, the size, which is thestandard of the touch acceptance frame with regard to the subject frame,is registered for each of the imaging modes.

After the process of step S33, the process progresses to step S34, thetouch acceptance frame setting processing section 33 references themovement detection section 32 and acquires the movement of the subject,and in step S35, it is determined whether or not it is necessary toadjust the touch acceptance frame set to the size which is the standard.

As described with reference to FIGS. 7 and 8 above, in the case wheremovement is detected where subjects approach each other, the touchacceptance frames are adjusted so that the spacing of the touchacceptance frames with regard to the subject frame on the opposing sidesof each of the subjects become narrower. In addition, in the case wheremovement is detected where subjects which are close to each otherseparate, the touch acceptance frame is adjusted so that the narrowedspacing is widened. Accordingly, the touch acceptance frame settingprocessing section 33 determines whether it is necessary to adjust thetouch acceptance frame in the case where movement of the subjects isdetected in this manner.

In a case where it is determined in step S35 that it is necessary toadjust the touch acceptance frame, the process progresses to step S36,the touch acceptance frame setting processing section 33 adjusts thetouch acceptance frame according to movement of the subject, and theprocess is completed.

On the other hand, in a case where it is determined in step S35 that itis not necessary to adjust the touch acceptance frame, the process skipsstep S36 and is completed.

As above, since the touch acceptance frame setting processing section 33sets the size of the touch acceptance frame according to the imagingmode, it is possible for the subject which is prioritized and imaged ineach of the imaging modes to be recognized as the tracking subject withpriority.

In addition, the touch acceptance frame setting processing section 33adjusts the touch acceptance frame according to the imaging scene (thatis, the context of the imaged image), more specifically, according tocharacteristics of the subject captured in the image (type, size, andthe like), movement of the subject relative to each other, or the like.According to this, it is possible to recognize the subject which isappropriate for each of the imaging scenes as the tracking subject withpriority.

In this manner, since the appropriate touch acceptance frame is setbased on the imaging mode and the imaging scene, it is possible torecognize the subject which the user wants as the tracking target withprecision.

In addition, in a case where the subjects move, there are cases wherethe position which the user touches and the position where the subjectis captured do not match, and there are cases where the subject, whichdiffers from the intention of the user, is recognized as the trackingsubject and the subject tracking process is performed. With regard tothis, in the digital camera 11, since the range of the touch acceptanceframe changes according to the movement of the subject, it is possibleto avoid the subject which differs from the intention of the user beingrecognized as the tracking subject and touching is not repeated aplurality of times in order for the user to select the correct subject.That is, since it is possible for the subject to be selected withprecision using a simple operation, it is possible to reduce the burdenon the user.

Here, it is possible for the digital camera 11 to be provided with theimaging mode other than the person mode and the pet mode describedabove, and for the subject frame setting processing section 24 and thetracking subject recognition section 25 to set the subject frame and thetouch acceptance frame for the subject which is prioritized in imagingfor each of the imaging modes. For example, the digital camera 11 isprovided with a child prioritizing mode which performs imaging withimaging conditions which are appropriate for imaging children, an adultprioritizing mode which performs imaging with imaging conditions whichare appropriate for imaging adults, or the like.

Screen P19 of the display section 14 when the child prioritizing mode isset as the imaging mode and screen P20 of the display section 14 whenthe adult prioritizing mode is set as the imaging mode are shown in FIG.12.

Screens P19 and P20 have the same configuration where two children andone adult are captured, subject frames F24 and F25 are displayed for thefaces of the children and a subject frame F26 is displayed for the faceof the adult. With regard to screens P19 and P20 such as these, there isa case where the position where the user touches the screen of thedisplay section 14 is between the subject frames F25 and F26 in the samemanner as the tip of the finger mark shown in screens P19 and P20.

In this case, if the imaging mode is the child prioritizing mode, thechild which is set with the subject frame F25 is recognized as thetracking subject, and if the imaging mode is the adult prioritizingmode, the adult which is set with the subject frame F26 is recognized asthe tracking subject. In this manner, in the digital camera 11, it ispossible for the intention of the user to be estimated by referencingthe imaging mode even when the touch position of the user is between thesubject frames F25 and F26 and for the tracking subject to be recognizedwith precision.

In addition, the digital camera 11 is provided with the imaging modessuch as an underwater mode, a food mode, a flower mode, or the like, thesubject frame is set with regard to the subject that is appropriate foreach of the imaging modes, and the touch acceptance frame is adjusted.That is, other than the face detector 28-1 and the animal detector 28-2shown in FIG. 1, it is possible for the subject frame setting processingsection 24 to be provided with a fish detector, a food detector, aflower detector, or the like. As the animal detector 28-2, there may bea dog face detector and a cat face detector. In addition, the digitalcamera 11 may be provided with a scenery mode, a night scene mode, orthe like as the imaging mode.

Here, other than being set by the user operating the digital camera 11,it is possible for the imaging modes to be automatically set so as toperform optimal imaging according to the imaging context by analyzingthe image in the digital camera 11.

In addition, in the embodiment, it is described that the face detector28-1 is normally operating, but there may be a state where the detectoris not normally operating in this manner and the subject frame is notdisplayed on the display section 14 before being touched by the user.That is, the operation of the detectors may be started at a timing whenthere is a touch by the user.

Here, in the embodiment, the subject to be tracked is selected by theuser touching the screen of the display section 14, but the subject maybe selected by another means, for example, by operation of operationkeys. In addition, the subject may be selected by using a guide framefor matching the subject with the angle of view in advance.

Here, other than the digital camera such as a digital still camera, adigital video camera, or the like, it is possible to apply thedisclosure to a device which is provided with an imaging function suchas a mobile phone device, a personal computer, or the like.

It is possible for the series of processes described above to beexecuted by using hardware and is possible for the series of processesdescribed above to be executed by using software. In a case where theseries of processes is executing using software, a program whichconfigures the software is installed from a program recording medium toa computer which has specialized hardware built in, or by installingeach type of program, to, for example, a generalized personal computerwhich is able to execute each type of function.

FIG. 13 is a block diagram illustrating a configuration example ofhardware of a computer which executes the series of processes describedabove using a program.

In the computer, a CPU (Central Processing Unit) 101, ROM (Read OnlyMemory) 102, and a RAM (Random Access Memory) 103 are connected to eachother by a bus 104.

In the bus 104, an input and output interface 105 is further connected.In the input and output interface 105, an input section 106 formed by akeyboard, a mouse, a microphone, or the like, an output section 107formed by a display, a speaker, or the like, a storage section 108formed by a hard disk, a nonvolatile memory, or the like, acommunication section 109 formed by a network interface or the like, adrive 110 which drives a magnetic disk, an optical disc, amagneto-optical disc, or a removable medium 111 such as a semiconductormemory or the like, are connected.

In the computer configured as above, the CPU 101 performs the series ofprocesses described above by, for example, executing the loading of theprogram stored in the storage section 108 in the RAM 103 via the inputand output interface 105 and the bus 104.

The program executed by the computer (CPU 101) is, for example, recordedin a magnetic disk (including a flexible disk), an optical disc (aCD-ROM (Compact Disc-Read Only Memory), a DVD (Digital Versatile Disc),or the like), a magneto-optical disc, or the removable medium 111 formedby a semiconductor memory or the like such as a package medium, or isprovided via a wired or wireless transmission medium such as a localarea network, the internet, or digital satellite broadcasting.

Then, it is possible to install the program in the storage section 108via the input and output interface 105 by mounting the removable medium111 in the drive 110. In addition, it is possible to receive the programusing the communication section 109 via the wired or wirelesstransmission medium and install the program in the storage section 108.Alternatively, it is possible to have the program installed in advancein the ROM 102 or the storage medium 108.

Here, the program executed by the computer may be a program whichperforms the processes in a time series in the sequence described in thespecifications or may be a program which performs the processes inparallel or at a necessary timing such as when a request is performed.In addition, the program may be processed using one CPU or may bedistributed and processed using a plurality of CPUs.

The present disclosure contains subject matter related to that disclosedin Japanese Priority Patent Application JP 2010-151951 filed in theJapan Patent Office on Jul. 2, 2010, the entire contents of which arehereby incorporated by reference.

It should be understood by those skilled in the art that variousmodifications, combinations, sub-combinations and alterations may occurdepending on design requirements and other factors insofar as they arewithin the scope of the appended claims or the equivalents thereof.

What is claimed is:
 1. An electronic device comprising: circuitryconfigured to control displaying an image on a display; detect a subjectin the image; control displaying, with the image, a subject area inwhich the subject detected from the image is included; set, but notdisplay, an acceptance area which encompasses the subject area accordingto a position of the subject area; detect a specified position which iscorresponding to a received user input position within the imagedisplayed with the subject area; detect whether the specified positionis within the acceptance area, and determine, in case the specifiedposition is within the acceptance area, the subject within the subjectarea corresponding to the acceptance area as a tracking subject.
 2. Theelectronic device of claim 1, further comprising: a touch panel displayconfigured to display the image and the subject area; and detect a touchinput as the user input.
 3. The electronic device of claim 1, whereinthe circuitry is configured to set a size of the acceptance areaaccording to a size of the subject detected in the image.
 4. Theelectronic device of claim 1, wherein the circuitry is configured to seta spacing of the acceptance area with regard to the subject areaaccording to a type of the subject detected in the image.
 5. Theelectronic device of claim 1, wherein the circuitry is configured toadjust a spacing of the acceptance area with regard to the subject areaaccording to movement of the subject detected in the image.
 6. Theelectronic device of claim 1, wherein the circuitry is configured toadjust a spacing of the acceptance area with regard to the subject frameaccording to a scene with regard to the subject.
 7. The electronicdevice of claim 1, wherein the circuitry is configured to adjust aspacing of the acceptance area with regard to the subject area accordingto relationship with another subject area.
 8. The electronic device ofclaim 1, wherein the circuitry is configured to set the acceptance areaof the subject which is appropriate for an imaging mode which iscurrently set to be larger than the acceptance area of the subject whichis not appropriate for the imaging mode when a plurality of types of thesubjects are detected in the image.
 9. The electronic device of claim 1,wherein the circuitry further configured to track the tracking subject.10. A method performed by an electronic device, the method comprising:controlling to display, by the circuity of the electronic device, animage on a display; detecting, by the circuity, a subject in the image;controlling, by the circuitry, with the image, to display a subject areain which the subject detected from the image is included; setting, butnot display, by the circuitry, an acceptance area which encompasses thesubject area according to a position of the subject area; detecting, bythe circuitry, a specified position which is corresponding to a receiveduser input position within the image displayed with the subject area;detect, by the circuitry, whether the specified position is within theacceptance area; and determine, in case the specified position is withinthe acceptance area, by the circuitry, the subject within the subjectarea corresponding to the acceptance area as a tracking subject.
 11. Themethod of claim 10, wherein a size of the acceptance area is setaccording to a size of the subject detected in the image.
 12. The methodof claim 10, wherein a spacing of the acceptance area with regard to thesubject area is set according to a type of the subject detected in theimage.
 13. The method of claim 10, wherein a spacing of the acceptancearea with regard to the subject area is adjusted according to movementof the subject detected in the image.
 14. The method of claim 10,wherein a spacing of the acceptance area with regard to the subjectframe is adjusted according to a scene with regard to the subject. 15.The method of claim 10, wherein a spacing of the acceptance area withregard to the subject area is adjusted according to relationship withanother subject area.
 16. The method of claim 1, wherein the acceptancearea of the subject which is appropriate for an imaging mode which iscurrently set to be larger than the acceptance area of the subject whichis not appropriate for the imaging mode is set when a plurality of typesof the subjects are detected in the image.
 17. The method of claim 1,the method further comprising tracking the tracking subject.
 18. Anon-transitory computer-readable medium including computer-programinstructions, which executed by an electronic device, cause theelectronic device to: control to display an image on a display; detect asubject in the image; control to display, with the image, a subject areain which the subjected detected from the image included; set, but notdisplay, an acceptance area which encompasses the subject area accordingto a position of the subject area; detect a specified position which iscorresponding to a received user input position within the imagedisplayed with the subject area; detect whether the specified positionis within the acceptance area; and determine, in case the specifiedposition is within the acceptance area, the subject within the subjectarea corresponding to the acceptance area as a tracking subject.